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About  the  Working  Paper  Series 


This  article  is  one  in  a  series  of  papers  addressing  one  or  more  issues  of  critical 
importance  to  the  acquisition  profession.  A  working  paper  is  a  forum  to  accomplish  a 
variety  of  objectives  such  as:  (1 )  present  a  rough  draft  of  a  particular  piece  of  acquisition 
research,  (2)  structure  a  “white  paper”  to  present  opinion  or  reasoning,  (3)  put  down 
one’s  thoughts  in  a  “think  piece”  for  collegial  review,  (4)  present  a  preliminary  draft  of  an 
eventual  article  in  an  acquisition  periodical,  (5)  provide  a  tutorial  (such  as  a  technical 
note)  to  accompany  a  case  study,  and  (6)  develop  a  dialogue  among  practitioners  and 
researchers  that  encourages  debate  and  discussion  on  topics  of  mutual  importance. 

A  working  paper  is  generally  the  “internal”  outlet  for  academic  and  research  institutions 
to  cultivate  an  idea,  argument  or  hypothesis,  particularly  when  in  its  infant  stages.  The 
primary  intent  is  to  induce  critical  thinking  about  crucial  acquisition  issues/problems  that 
will  become  part  of  the  acquisition  professional  body  of  knowledge. 

It  is  expected  that  articles  in  the  working  paper  series  will  eventually  be  published 
in  other  venues  such  as  articles  in  refereed  journals  and  other  periodicals,  as  technical 
reports,  as  chapters  in  a  book,  as  cases  or  case  studies,  as  monographs,  or  a  variety  of 
other  similar  publications. 

Readers  are  encouraged  to  provide  both  written  and  oral  feedback  to  working 
paper  authors.  Through  rigorous  discussion  and  discourse,  it  is  anticipated  that 
underlying  assumptions,  concepts,  conventional  wisdom,  theories  and  principles  will  be 
challenged,  examined  and  articulated. 
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Reliability  isn’t  everything;  it  is  the  only  thing! 


At  the  present  time,  the  most  serious  problem  in  logistics  support  for  the  life  of  a 
weapon  system  is  the  asymmetry  of  the  demonstrated  reliability  of  components  and  the 
inventory  and  maintenance  infrastructure  necessary  to  match  that  reliability. 

Reliability  is  the  single  most  dominant  life  cycle  cost  driver  and  is  the  key  enabler  of 
acceptable  cost  effective  operational  availability.  The  greater  the  time  between  failures 
of  components,  the  less  we  require  expensive  maintenance,  critical  test  equipment, 
unique  training  and  high  priced  inventories  as  well  as  other  logistics  elements.  The 
DoD  and  the  Navy  are  struggling  with  the  results  of  the  imbalance  of  poor  inherent 
reliability  of  components  on  the  one  hand,  and  the  consequences  of  highly  exaggerated 
reliability  figures  of  merit  used  for  life  cycle  support  planning  on  the  other.  The  DoD  and 
the  Navy  have  not  understood  the  results  of  a  continuing  failure  to  properly  acquire, 
measure,  manage  and  support  demonstrated  reliability.  We  simply  have  too  many 
demands  for  too  few  spare  parts  because  of  this  asymmetry. 

For  the  sake  of  a  common  reference,  let’s  define  reliability.  Reliability  is  comprised  of 
four  components:  probability,  satisfactory  performance,  time,  and  specified  operating 
conditions.  Taking  these  four  elements  together,  we  define  reliability  as  the  probability 
that  a  system,  component,  or  part  will  operate  satisfactorily  for  a  specified  period  of  time 
under  specified  operating  conditions. 

Probability  in  this  definition  refers  to  a  quantitative  expression  representing  a  percent 
specifying  the  number  of  hours  we  can  expect  a  system  to  operate  satisfactorily  when 
we  operate  it.  For  example,  if  we  state  that  the  probability  of  satisfactory  performance 
for  a  hydraulic  actuator  for  100  hours  is  .8,  then  we  can  expect  the  system  to  survive 
100  hours  80%  of  the  time.  In  an  inventory  of  like  hydraulic  actuators  we  can  expect  the 
same  probability  of  survival,  but  experience  shows  us  that  failures  will  occur  at  different 
times  in  a  probabilistic  manner. 
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Satisfactory  performance  relates  to  the  specific  criteria,  which  describes  proper 
performance,  i.e.  operate  a  trailing  edge  flap  when  a  control  input  is  made. 

Time  is  the  key  consideration  when  referring  to  reliability.  Time  is  how  we  measure  the 
probability  of  completing  a  mission  or  how  often  we  have  to  do  maintenance  or  gauging 
satisfactory  performance  with  respect  to  time  for  spares  inventory-planning  purposes. 
We  commonly  define  reliability  in  terms  of  mean  time  between  failure  (MTBF),  mean 
time  between  maintenance,  (MTBM)  and  mean  time  to  failure,  (MTTF).  It  follows  that 
the  more  frequent  the  failures,  the  greater  the  number  of  spares  required  as  well  as 
increased  requirements  for  all  the  other  logistics  elements. 

Specified  operating  conditions  define  the  way  a  system  or  component  will  be  used, 
the  environment  it  will  be  used  in,  and  includes  storage,  packaging,  handling  and 
transportation. 

The  qualities  of  each  of  the  four  elements  of  reliability  result  from  a  design-requirement 
synergy.  The  resulting  reliability  is  an  inherent  quality  of  that  design.  That  design 
produces  a  component  that  has  a  physically  constrained  reliability;  an  inherent  reliability 
that  is  the  best  we  can  achieve  in  an  ideal  operating  and  maintenance  environment. 

I  have  heard  maintenance  officers  say  that  they  can  improve  the  reliability  of  a 
component  by  improving  the  maintenance.  That  simply  is  not  possible  for  the  reason  I 
just  stated.  In  1982  I  participated  in  a  study  at  NAVAIR  that  showed  that  the  only  way  to 
improve  the  inherent  reliability  of  a  system  is  to  change  the  technology  or  physical 
construct  of  that  system.  The  inertial  platform  that  was  used  in  the  A-6E  is  a  good 
example  of  this  concept.  Initially,  the  platform  consisted  of  3  mechanical  gyros,  one 
each  for  the  X,  Y,  and  Z  axes  with  MTBFs  in  the  low  two-digits  and  in  its  final 
configuration  with  ring-laser  gyros,  the  failure  rate  was  three  to  four  figures. 
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Some  additional  reliability  measures  to  consider: 


Failure  rate  X=  number  of  failures 
Total  operating  hours 

MTBF =  J_ 

X 

For  illustrative  purposes  we  will  assume  a  reliability  function  in  terms  of  a  Poisson 
distribution  thus  reliability  or  the  probability  of  survival  is  expressed  as 

R(t)  =  e™ 

Where  R  is  reliability 
e  is  the  natural  log  base  (2.7182) 

K  is  the  number  of  items  used  of  a  particular  type 
X  is  the  failure  rate  (1/MTBF) 
t  is  the  time  period  of  interest 
Some  factors  related  to  reliability  to  consider: 

Inherent  availability-  is  the  probability  that  a  system  when  used  under  specified 
conditions  in  an  ideal  environment  (i.e.  specified  operating  conditions,  properly  trained 
technicians,  spares  at  the  ready,  tools  etc.)  will  operate  satisfactorily  at  any  time 
required.  This  definition  excludes  scheduled  maintenance,  logistics  delay  time,  and 
administrative  delay  time. 

The  expression  is-  Ai  =  MTBF 

MTBF  +  Met 

Where  MTBF  is  the  mean  time  between  failure  and  Met  is  the  mean  corrective 
maintenance  time  or  mean  time  to  repair. 
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Achieved  availability-  is  the  probability  that  a  system  operated  and  supported  under 
specified  conditions  in  an  ideal  support  environment  as  above  will  perform  as  required 
at  any  time.  Achieved  availability  includes  scheduled  maintenance  but  excludes 
logistics  delay  time  and  administrative  delay  time. 

The  expression  is-  Aa  =  MTBM _ 

MTBM  +  M 

Where  MTBM  is  the  mean  time  between  maintenance  and  is  the  mean  active 
maintenance  time. 

Operational  availability-  is  the  probability  that  a  system  when  used  under  specified 
conditions  in  an  actual  operational  environment  will  operate  satisfactorily  when  required. 

The  expression  is  -  A0  =  MTBM _ 

MTBM  +  MDT 

Where  MTBM  is  as  above  and  MDT  is  the  mean  maintenance  down  time.  The 
reciprocal  of  MTBM  is  the  frequency  of  maintenance  that  includes  scheduled  and 
unscheduled  maintenance.  The  mean  time  between  unscheduled  maintenance  should 
be  ~  MTBF. 

Spare  part  quantity  determination  is  a  function  of  a  probability  of  having  a  spare  part 
when  needed,  the  reliability  of  the  item  in  question  and  the  quantity  of  items  used  in 
the  system.  (It  is  significant  to  point  out  that  the  F/A-18  item  manager  at  NAVICP  is 
using  MTBD,  meantime  between  demands,  for  inventory  determination  that  in 
application  bypasses  the  source  of  the  problem) 
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The  expression  is  - 


P  =  Z 
n=0 


R(1-lnR)n 

L  n!  J 


where 

P  =  the  probability  of  having  a  spare  of  a  particular  item  available  when  required 
S  =  the  number  of  spare  parts  carried  in  stock 
R  =  composite  reliability  as  stated  above 
K  =  quantity  of  parts  used  of  a  particular  type 
InR  =  natural  logarithm  of  R 

Now  that  we  have  common  references,  let’s  examine  the  concerns  about  properly 
acquiring,  measuring  and  managing  reliability  figures  of  merit  as  the  key  parameter  of 
providing  life  cycle  support. 

First,  we  will  consider  how  we  acquire  a  reliability  figure  of  merit  for  a  component. 
Typically,  a  vendor  will  submit  a  reliability  measure  based  on  testing,  estimates, 
expected  reliability  growth  etc.  Since  operational  Test  and  Evaluation  is  expensive  and 
focuses  on  systems  rather  than  individual  components,  component  unreliability  may  go 
unexposed.  For  the  lack  of  any  other  data  the  logistics  managers  tend  to  accept  the 
contractor’s  claim  of  X  hours  MTBF.  (It  is  noted  here  that  with  a  paucity  of  data,  it  would 
serve  us  well  to  examine  the  history  of  performance  of  like  items  already  in  service 
being  used  in  a  similar  way)  Based  on  that  reliability  figure  of  merit,  it  is  then  applied  to 
our  spare  part  quantity  calculation  that  yields  the  number  of  spares  we  should  carry. 

(for  illustration  purposes,  we  are  ignoring  component  cost,  operational  scenario  etc.) 

As  an  example  we  will  consider  the  Trailing  Edge  Flap  Actuator  (TEF)  for  the  F/A-18  A- 
D.  In  establishing  initial  support  for  the  TEF  we  set  its  reliability  figure  of  merit  at  4000 
(According  to  NAVAIR  APML  circa  1997)  hours  mean  time  between  failure.  From  the 
above  we  show  that  the  failure  rate  is  .0025  in  this  case.  Now  let’s  assume  we  have  an 
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airplane  inventory  of  200  airplanes,  each  of  which  is  to  operate  30  hours  per  month  and 
we  replenish  stock  every  90  days.  Applying  this  failure  rate  to  our  equation  to 
determine  our  spare  parts  requirement  we  will  use  the  following: 

P  (protection  level)  is  the  Probability  of  having  the  part  on  hand  when  required  in 
this  case  assumed  to  be  .95 

K  =  400  parts  (2  per  airplane) 

S  =  the  number  of  spares  to  be  determined 

R  =  reliability  R  =  e-kM 

LnR  =  natural  log  of  R 

X  =  .00025  failures  per  hour 

T  =  the  stock  replenishment  cycle  of  90  days  or  3  months 

kA,t=  number  of  items  X  the  failure  rate  X  operating  time  per  airplane  X  stocking 
intervals 

thus: 

kXt  =  400(.00025)(30)(3)  or  400  parts  operated  30  hours  per  month  for  3  months 
at  a  failure  rate  of  .00025  =  9 

To  facilitate  solving  our  equation  we  use  the  NAVSHIPS  94324  nomograph  and  enter 
the  kM  value  of  9  and  refer  to  the  P  value  of  .95  and  we  get  14  spares  are  required. 

If  we  assume  a  protection  level  of  .85,  10  spares  are  required 

Taking  the  demonstrated  failure  rate  we  solve  our  equation  again.  According  to  the 
latest  APML  reliability  figure  of  merit  we  have  an  MTBF  of  138  hours  for  the  TEF  (Cdr. 
Ellen  Coyne,  NAVAIR,  email  20  July  1999) 

Now,  X  =  1/138  =  .00722  per  hour.  It  follows  that  kXt  =  400(.00722)(30)(3)  =  259.9. 

Now  we  need  ~  300  spares.  We  have  a  situation  where  the  failure  rate  is  29  times  that 
predicted  and  the  spares  on  hand  are  ~  1/20th  that  required. 
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The  F/A-18  item  manager  states  the  TEF  is  presently  performing  at  an  MTBD  of  900 
hours.  If  we  assume  an  inventory  calculation  substituting  MTBD  for  MTBF  we  derive 
the  following: 

A,  =  .0011 

kXt  =  400(.0011)(30)(3)  =  39.6 

As  in  the  above  procedure  we  have  a  failure  rate  4.3  times  predicted  and  we 
require  50  spares  which  is  ~3.5  times  predicted. 

The  point  here  is  that  whichever  reliability  is  correct,  significant  asymmetry  exists. 

The  asymmetry  shown  in  this  example  is  one  of  many  that  have  impeded  the  F/A-18 
from  achieving  its  inherent  availability  (A)  since  its  introduction.  We  have  a  pattern  of 
failure  rates  that  far  exceed  unsubstantiated  levels  that  have  been  used  to  provision 
support  for  the  F/A-18  and  other  systems  resulting  in  under-budgeting  logistics  support, 
cannibalization  and  its  costs,  increased  workload  on  maintenance  personnel,  potential 
safety  risks  and  most  significantly  an  operational  readiness  potential  that  is  unrealized. 
Although  we  know  that  the  demonstrated  reliability  is  not  what  was  predicted  we  have 
not  recomputed  the  spares  required  and  have  not  made  the  necessary  provisioning 
corrections  and  investments.  Moreover  we  tend  not  to  conduct  a  follow-on  Level  of 
Repair  Analysis  (LORA). 

How  can  we  improve  the  establishment  of  reliability  figures  of  merit  for  the  purposes  of 
spare  inventory  symmetry?  First,  we  should  adopt  a  null  hypothesis,  which  states  that  a 
claim  of  a  reliability  figure  of  merit  for  a  given  component  is  not  true  until  proven  by  the 
contractor.  (The  Navy  should  verify  contractor  MTBFs  in  a  DT/OT  continuum)  If  the 
contractor’s  initial  claim  cannot  be  established,  then  we  next  ask  what  value  can  be 
proved  and  is  that  value  acceptable?  Once  a  value  has  been  proved,  then  that  is  the 
value  that  should  drive  the  spare  inventory  to  support  that  given  component. 

Our  experience  in  Naval  Aviation  has  shown  that  reliability  declines  over  time,  but  our 
support  analyses  are  not  recomputed  to  match  those  changes.  It  follows  that  logistics 
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managers  and  sustainment  engineers  should  recompute  support  requirements  in  light  of 
declining  reliability  and  that  budgets  must  be  adjusted  to  support  the  derived 
requirements. 

We  have  the  tools  at  our  disposal  to  correct  the  asymmetry  of  reliability  and  spares 
inventories  for  our  weapon  systems.  We  should  strive  for  accuracy  in  establishing 
reliability  measures  for  new  programs.  In  order  to  ensure  we  have  the  best  measure  of 
reliability,  we  should  employ  the  Null  Hypothesis  that  says  the  contractor  must  prove 
claims  of  performance  and  not  the  Navy.  Once  a  figure  of  merit  is  demonstrated  we 
must  ensure  the  inventory  matches  the  reliability.  Throughout  the  life  of  a  system  we 
should  continuously  analyze  reliability  performance  and  recompute  spare  parts 
inventories  based  on  our  analysis  and  finally  we  must  make  the  financial  commitment  to 
make  these  efforts  successful. 
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August  2004 

NPS-LM-04-003  Enablers  to  Ensure  a  Successful  Force  Centric  Logistics  Enterprise 
April  2004 

Acquisition  Symposium  Proceedings 


NPS-AM-04-005  Charting  a  Course  for  Change:  Acquisition  Theory  and  Practice  for 
a  Transforming  Defense 
May  2004 


Copies  of  the  Acquisition  Sponsored  Research  Reports  may  be  printed  from  our 
website  www.nps.navv.mil/qsbpp/acgn/publications 
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FY  2003  Sponsored  Acquisition  Research  Products 


Sponsored  Report  Series 

NPS-AM-03-003  Centralized  Control  of  Defense  Acquisition  Programs: 

A  Comparative  Review  of  the  Framework  from  1987  -  2003 
September  2003 

NPS-AM-03-004  Reduction  of  Total  Ownership  Cost 
September  2003 

NPS-CM-03-006  Auto-Redact  Toolset  for  Department  of  Defense  Contracts 
September  2003 

Working  Paper  Series 

NPS-CM-03-002  Transformation  in  DOD  Contract  Closeout 
June  2003 

Acquisition  Case  Series 

NPS-CM-03-005  Contract  Closeout  (A) 

September  2003 

Other  Sponsored  Research 

NPS-CM-03-001  Transformation  in  DOD  Contract  Closeout 
MBA  Professional  Report 
June  2003 


Copies  of  the  Acquisition  Sponsored  Research  Reports  may  be  printed  from  our 
website  www.nps.navv.mil/qsbpp/acgn/publications 
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